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ASSESSMENT  OF  TWO  COMPUTER  BASED  PRODUCTS:  THE  MILITARY 
DECISION-MAKING  PROCESS  AND  THE  BRIGADE  BATTLE  CAPTAIN 

EXECUTIVE  SUMMARY 


Research  Requirements: 

In  response  to  frequently  identified  weaknesses  in  the  ability  of  battle  staff 
members  to  perform  under  the  stress  of  Combat  Training  Center  (CTC)  missions, 
two  computer-based  courses  were  developed  by  the  U.S.  Army  Research 
Institute  (ARI),  Infantry  Forces  Research  Unit.  One  course  contains  material 
designed  to  train  battle  staff  members  in  the  Military  Decision-Making  Process 
(MDMP)  and  the  other  course  was  designed  to  train  Battle  Captains.  The  two 
courses  contain  both  doctrinal  material  and  tactics,  techniques,  and  procedures 
(TTPs)  provided  by  personnel  from  the  Joint  Readiness  Training  Center’s  (JRTC) 
Leaders  Training  Program  (LTP). 

The  purpose  of  the  research  was  to  perform  an  assessment  of  the 
effectiveness  of  the  MDMP  and  Brigade  Battle  Captain  courses.  Because  both 
courses  contain  materials  from  doctrinal  manuals  as  well  as  TTPs  gathered  from 
lessons  learned,  the  evaluation  examined  the  programs  from  both  perspectives. 
The  intent  of  the  assessment  was  to  provide  a  foundation  for  building  better 
training  materials  in  the  future,  and  to  make  any  improvements  to  these  two 
prototype  products. 

Procedure: 

Volunteer  officers  from  Active  Duty,  National  Guard,  and  Army  Reserve 
units  participated  in  the  assessment  of  the  two  courses.  Officers  were  given  a 
CD  containing  the  program  and  instructed  to  complete  the  course  at  their  own 
pace.  After  completion  of  the  course,  the  test  subject  officers  filled  out  an 
evaluation  form  designed  to  provide  the  researcher  with  their  opinions  on  the 
effectiveness  of  the  course  materials. 

Findings: 

Both  courses  were  well  received  by  the  officers  participating  in  the 
evaluations.  More  than  75%  rated  the  lessons  and  sub  lessons  as  being 
“generally  effective”  or  very  effective.”  The  TTPs  included  in  the  courses  were 
viewed  as  the  most  beneficial  aspects  of  the  courses.  The  test  officers  also 
provided  opinions  on  the  course  content  and  course  design.  For  course  content, 
the  officers  cited  areas  in  need  of  more  material  (e.g.,  Emergency  Jump  TOC 
Procedures)  or  less  material  (e.g.,  Issue  Warning  Order).  The  most  frequent 
course  design  request  was  for  a  bookmark  feature  that  would  allow  quick  return 


to  an  uncompleted  lesson.  However,  for  the  most  part,  the  test  officers  indicated 
that  the  courses  were  effective  training  vehicles. 

Utilization  of  Findings: 

The  design  of  future  courses  can  be  enhanced  based  on  the  feedback 
elicited  from  the  test  officers.  In  addition,  the  course  content  can  be  changed  in 
any  upgrades  to  these  prototype  training  programs  to  better  reflect  the  needs  of 
officers  filling  battle  staff  positions.  ARI  has  received  many  requests  for  the 
materials  in  their  current  form  and  they  are  being  distributed  to  battalion  and 
brigade  staff  officers  prior  to  attendance  at  JRTC’s  LTP. 
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Assessment  of  Two  Computer  Based  Products: 

The  Military  Decision-Making  Process  and  The  Brigade  Battle  Captain 


Background 

In  the  early  1990s,  the  U.  S.  Army  Research  Institute  (ARI)  conducted  several 
large-scale  research  programs  at  the  Combat  Training  Centers  (CTCs).  Much  of  the 
work  reported  concentrated  on  Infantry  battalions  and  the  relationship  between  the 
amount  and  content  of  their  home  station  training  and  their  subsequent  performance  at 
either  the  National  Training  Center  (NTC)  or  the  Joint  Readiness  Training  Center 
(JRTC).  (See  Holz,  Hiller  and  McFann,  1994,  for  a  comprehensive  overview  of  this 
research  on  measuring  and  managing  unit  training  readiness.) 

A  recurrent  and  somewhat  unexpected  finding  or  byproduct  of  this  research 
program  was  that  many  battalion  and  brigade  staff  officers,  apparently  capable  of 
performing  their  jobs  at  home  station,  do  not  do  well  under  the  stress  of  a  CTC  rotation. 
As  reported  in  Battle  Staff  Training  and  Synchronization  in  Light  Infantry  Battalions  and 
Task  Forces  (Thompson,  Thompson,  Pleban,  &  Valentine,  1991),  performance 
shortfalls  by  battle  staffs  were  widespread,  and  appeared  to  stem  in  large  part  from  a 
lack  of  training  programs  and  materials  for  staff  officers.  Staffs  did  not  know  their  jobs, 
and  staff  performance  was  not  synchronized. 

While  Officer  Advance  Course  training  formerly  covered  materials  related  to 
functional  area  training,  two  things  have  changed.  First,  the  staff-specific  material  that 
used  to  be  taught  in  Advance  Courses  has  been  largely  eliminated,  and  more 
importantly,  present  day  officers  typically  have  already  had  several  battalion  or  even 
brigade  staff  positions  before  their  attendance  at  an  Advance  Course.  As  a  result,  they 
have  normally  had  very  little  formal  training  related  to  their  own  individual  jobs.  The 
finding  that  many  officers  have  held  staff  positions  without  benefit  of  relevant  training, 
also  confirmed  by  Salter  (1994),  goes  across  branches  and  jobs.  Typically  only  the 
motor  officer  or  the  S2  (Intelligence  Officer)  has  had  formal  training  in  his  “specialty.” 
The  rest  of  the  staff  members,  especially  at  battalion  level,  learn  their  jobs  as  they 
experience  them. 

Performance  at  home  station  is  adequate  when  stress  is  limited.  Many  activities 
have  sufficient  lead-time  to  permit  relatively  leisurely  on  the  job  training.  The  fast  paced 
CTC  rotations,  however,  permit  little  recovery  time,  and  exact  a  greater  penalty  for 
errors  (Thompson,  et  al.,  1991,  and  Thompson,  Pleban  and  Valentine,  1994).  A  staff 
officer  who  is  not  thoroughly  versed  in  his  job  will  be  unable  to  perform  under  such 
pressure.  Numerous  articles  from  the  bulletins  and  newsletters  produced  under  the 
auspices  of  the  Center  for  Army  Lessons  Learned  (CALL)  at  Fort  Leavenworth,  Kansas, 
confirmed  this  research. 
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Preliminary  Solutions 


Because  evidence  suggested  that  many  staff  officers  lack  the  basic  knowledge 
necessary  to  perform  their  jobs,  an  obvious  first  step  toward  improving  performance 
was  to  create  materials  outlining  required  staff  functional  duties  and  responsibilities. 

The  Commander’s  Battle  Staff  Handbook  (Pleban,  Thompson,  &  Valentine,  1993)  was 
designed  to  assist  staff  members  with  these  skills  and  as  a  tool  to  organize  staff 
training.  The  intent  of  this  product  was  to  provide  staff  officers  a  quick  reference  that 
would  serve  as  a  guide  toward  their  overall  responsibilities. 

Another  attempt  to  provide  detailed  knowledge  based  training  to  young  officers 
came  from  a  series  of  projects  sponsored  by  the  Defense  Advance  Research  Projects 
Agency  (DARPA).  Many  of  these  DARPA  sponsored  training  programs  were  focused 
on  personnel  in  the  Army  National  Guard.  One  of  the  first  training  programs,  individual 
computer-based  and  text-based  instruction  was  developed  by  ARI  at  Fort  Benning,  and 
distributed  in  a  prototype  version  first  for  Infantry  battalion,  then  for  brigade  staffs  (Andre 
and  Salter,  1995, 1996a,  1996b,  1996c).  These  programs  provided  a  staff  operations 
overview  coupled  with  individual  doctrinally  based  instruction  in  each  of  thirteen  staff 
positions.  Included  are  interactive  software  and  text  supporting  materials  for  the  XO, 

SI ,  S2,  S3,  S3  Air,  S4,  Fire  Support  Officer,  Signal  Officer,  Engineer,  Chemical  Officer, 
Air  Defense  Artillery  Officer,  and  Chaplain.  An  additional  Common  Core  Course 
provides  information  that  pertains  to  all  staff  positions.  Available  on  floppy  disc,  this 
program  provided  preliminary  structured  and  standardized  initial  training  for  individual 
staff  personnel. 

This  introductory  effort  was  refined,  adapted  and  expanded  by  ARI  at  Fort  Knox 
to  comprise  a  set  of  training  materials  specifically  geared  toward  students  at  the  Armor 
Advance  Course.  This  work,  collectively  referred  to  as  the  Battle  Staff  Training  System 
(BSTS),  is  reported  in  Andre,  Wampler  and  Olney  (1997).  The  BSTS  consists  of  a 
combination  of  text  and  computer  based  materials,  together  with  end  of  lesson  checks 
on  learning  and  end  of  course  tests.  Like  the  original  DARPA  sponsored  National 
Guard  version,  BSTS  has  paper-based  materials,  multiple  floppy  discs,  and  an 
embedded  training  management  system.  Individual  soldiers  receive  self-paced 
instruction  on  the  requirements  of  their  staff  jobs.  A  follow  on  to  this  research, 
conducted  by  ARI  under  the  Force  XXI  Training  Program,  expanded  the  individual 
training  modules,  added  the  Commander  position,  and  provided  a  greater  proportion  of 
computer  lessons,  to  include  the  Common  Core,  totally  computer-based.  A  full 
description  can  be  found  in  Wampler  and  Livingston  (1998). 

The  goal  of  all  of  these  training  materials,  whether  computer-based  or  paper 
products,  was  to  provide  staff  officers  with  the  basic  required  information  necessary  to 
perform  their  specific  staff  jobs.  The  training  materials  are  doctrinally  based,  and  are 
geared  toward  individual  instruction  for  each  of  the  battalion  and  brigade  staff  positions. 
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Problem 


However,  this  material  which  provides  individual  staff  position  content,  is  only  the 
first  step  in  training  staff  officers.  The  BSTS  versions  allow  little  interaction  with  other 
staff  officers.  That  is,  the  training  is  mostly  knowledge  based  and  geared  to  teach 
individual  primary  job  responsibilities.  Still  missing  from  these  first  attempts  at  staff 
training  were  opportunities  for  the  staff  officers  to  interact  with  other  staff  officers  to 
practice  their  craft.  Also,  these  programs  had  very  limited  tactics,  techniques,  and 
procedures  (TTPs)  -  the  “how  to”  of  the  tasks.  Once  staff  members  know  and 
understand  their  primary  duties  and  responsibilities,  they  must  work  together  in  planning 
and  executing  their  battlefield  functions. 

The  Military  Decision-Making  Process 

The  Military  Decision-Making  Process  (MDMP)  is  one  vehicle  which  allows  staff 
members  to  work  together  to  achieve  their  mission.  The  MDMP  is  the  thought  process 
used  for  examining  the  battlefield.  It  is  an  adaptation  of  the  Army’s  analytical  approach 
to  problem  solving  -  a  single,  established,  and  proven  process.  It  comprises  a  series  of 
actions,  changes,  or  functions  that  achieve  an  end  result.  Each  step  begins  with  certain 
input  that  builds  upon  the  previous  steps.  The  MDMP  thus  consists  of  a  series  of  well- 
documented  steps  which,  if  followed,  permit  a  staff  to  effectively  and  efficiently  plan  for 
a  mission.  The  MDMP,  however  requires  not  only  that  the  officer  knows  his  own  job, 
but  how  that  job  interacts  with  and  in  turn  depends  on  the  activities  performed  by  other 
staff  members.  In  other  words,  successful  implementation  of  the  MDMP  requires 
coordination. 

CALL  Bulletins  and  Newsletters  have  repeatedly  identified  staff  problems  in 
execution  of  the  MDMP  and  specific  deficiencies  in  staff  planning  (i.e.,  steps  of  the 
MDMP).  In  CTC  Bulletin  93-4  (1993)  weaknesses  in  synchronization  matrix 
development  and  course-of-action  (COA)  development  were  reported.  Additional 
evidence  that  training  needs  existed  across  all  battlefield  operating  systems  for  MDMP 
was  reported  in  CTC  Trends  Joint  Readiness  Training  Center  4QFY95  and  1QFY96 
(1995)  and  NTC  Priority  Trends  (1996).  In  most  cases,  even  when  individuals  knew 
their  own  jobs,  they  were  unaware  of  how  to  integrate  them  with  the  rest  of  the  staff. 

Therefore,  training  which  contains  only  individual  staff  member  information  is  not 
enough.  Even  after  the  preliminary  ARI  training  products,  especially  the  Commander’s 
Battle  Staff  Handbook  (Pleban,  Thompson  and  Valentine,  1993)  became  widely 
available  at  the  CTCs,  reports  of  deficiencies  in  MDMP  steps  were  widespread  (e.g., 
CTC  Trends  JRTC  2QFY97  &  3QFY97,  1997;  CTC  Trends  98-14,  1998b;  CTC 
Quarterly  Bulletin  98-12,  1998a).  These  deficiencies  indicate  that  there  is  a  definite 
need  for  training  that  includes  TTPs  and  the  basics  of  staff  interaction. 

Personnel  from  the  JRTC  emphasized  this  finding.  The  JRTC  Leaders  Training 
Program  (LTP)  requires  brigade  and  battalion  staffs  to  respond  to  an  order  by 
developing  a  plan  -  i.e.,  by  executing  the  MDMP.  Personnel  from  the  LTP  noted  that 
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brigade  and  battalion  staff  personnel  often  had  difficulty  in  implementing  the  MDMP.  At 
the  LTP  it  was  apparent  that  many  staff  officers  still  did  not  know  their  jobs,  and  of  those 
that  did,  many  had  had  little  experience  in  integrating  their  jobs  with  the  rest  of  the  staff. 
Too  many  brigade  and  battalion  staffs  were  unaware  of  how  actually  to  execute  the 
MDMP. 

To  respond  to  that  expressed  need,  to  have  staffs  better  versed  in  the  MDMP, 
ARI  developed  the  first  of  several  compact  disc  (CD)  computer-based  programs.  The 
Military  Decision-Making  Process  course  is  totally  computer  based  (no  text)  and  is 
administered  through  a  self-loading  CD  program.  Developed  in  concert  with  the  JRTC 
LTP  personnel,  the  MDMP  follows  the  seven  step  MDMP  process:  Receipt  of  Mission, 
Mission  Analysis,  Course  of  Action  Development,  Course  of  Action  Analysis 
(Wargaming),  Course  of  Action  Comparison,  Course  of  Action  Approval,  and  Orders 
Production.  The  MDMP  course  targeted  selected  steps  of  the  MDMP,  and  provided 
useful  TTPs,  not  only  for  the  entire  staff,  but  selected  TTPs  for  the  commander,  the  XO, 
SI,  S2,  S3,  S4,  Fire  Support  Officer,  the  Aviation  Commander,  and  combat  support 
personnel.  The  lesson  content  of  the  MDMP  was  adopted  from  Field  Manual  (FM)  101- 
5,  Staff  Organization  and  Operations  (DA,  1997).  The  unique  information,  material  not 
found  in  the  FM,  comprises  the  TTPs,  which  are  incorporated  into  each  of  the  lessons. 
TTPs  from  JRTC  lessons  learned  and  the  LTP  are  contained  in  the  lessons  for  the 
appropriate  staff  members.  Fuller  description  of  the  CD  based  MDMP  program, 
reviewed  and  administered  by  the  JRTC,  is  reported  in  Wampler,  Centric  and  Salter 
(1998b). 

Information  Management 

Along  with  reported  breakdowns  in  the  MDMP  process,  CALL  Newsletters  and  Bulletins 
have  documented  problems  in  battle  tracking  and  information  management  (e.g.,  NTC 
Priority  Trends,  CALL,  1996  and  CTC  Trends  JRTC,  CALL,  1997).  These  problems 
appear  to  be  common  across  battlefield  operating  systems.  Commanders  do  not 
receive  the  information  necessary  to  see  the  battlefield  in  a  timely  manner,  or  in  the 
level  of  detail  they  need. 

In  order  for  a  staff  to  properly  execute  their  mission,  battlefield  information  must 
be  tracked,  analyzed,  and  acted  upon.  The  Tactical  Operations  Center  (TOC)  serves 
as  the  unit’s  information  center.  For  successful  mission  execution,  the  TOC  must 
process  an  enormous  number  of  messages,  reports,  and  orders.  All  information  from 
these  sources  must  be  analyzed  and  acted  upon  by  alerting  the  appropriate  staff 
sections.  Members  of  the  TOC  must  know  their  responsibilities  and  what  actions  to 
take  based  on  the  available  information.  Actions  may  include  prioritizing  information, 
decision  making,  and  informing  other  sections.  The  key  staff  member  to  help 
synchronize  this  management  of  information  is  the  Battle  Captain  (Me  Williams,  1998). 

The  Battle  Captain  is  responsible  for  deciding  where  incoming  information  must 
go  to  be  processed.  The  Battle  Captain  must  conduct  a  quick  analysis  to  ensure  the 
data  are  timely  and  accurate.  The  analysis  should  include  who  gets  the  information, 
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what  should  be  done  to  it,  and  how  it  will  impact  on  the  mission.  In  other  words,  the 
Battle  Captain  serves  as  a  crisis  management  supervisor  (Me  Williams,  1998).  As 
important  as  the  position  of  Battle  Captain  is  to  the  TOC,  there  are  no  existing  formal 
doctrinal  publications  for  this  job,  and  there  is  no  Table  of  Organization  and  Equipment 
(TO&E)  position  labeled  Battle  Captain.  However,  units  understand  the  importance  of 
the  position  and  every  TO&E  unit  uses  a  Battle  Captain  -  an  officer/NCO  to  coordinate 
and  supervise  TOC  activities  (Smith,  1997). 

The  ideal  individual  selected  to  be  the  Battle  Captain  would  be  a  senior  captain 
who  has  already  completed  a  company  command.  More  likely,  however,  the  Battle 
Captain  is  a  senior  lieutenant  or  very  junior  captain  who  has  recently  graduated  from  the 
advanced  course.  He  (or  she)  may  have  little  or  no  time  in  a  staff  job,  and  in  many 
cases,  is  the  newest  member  of  the  staff.  Since  there  are  no  formal  doctrinal 
publications  to  train  Battle  Captains,  and  because  individuals  serving  as  Battle  Captains 
have  limited  experience,  a  need  exists  for  standardized  doctrinally  based  training 
material  geared  for  the  Battle  Captain.  The  JRTC  has  incorporated  a  Battle  Captain 
Course  into  the  LTP  to  insure  that  the  officer  selected  for  this  critical  position  is  aware  of 
the  job’s  responsibilities.  However,  this  material  is  available  only  to  designated  Battle 
Captains,  already  in  the  job  (Smith,  1997).  Nothing  is  available  for  an  individual  or  a 
unit  to  study  in  advance,  to  prepare  for  the  job. 

To  address  some  of  these  identified  Battle  Captain  problems,  ARI  again  created 
a  CD  based  training  program,  to  assist  the  new  or  potential  Battle  Captain  in  the 
performance  of  his  job.  Much  of  the  material  came  from  the  CALL  bulletins  and 
newsletters;  a  great  part  came  from  the  JRTC  LTP  materials  (Smith,  1997).  The 
genesis  of  the  Brigade  Battle  Captain  Course  and  a  full  description  can  be  found  in 
Wampler,  Centric  and  Salter  (1998a). 

Target  Audience  for  the  MDMP  and  Brigade  Battje  Captain  Programs 

The  ARI  developed  training  packages  were  originally  designed  for  and  in 
conjunction  with  the  JRTC.  Since  the  MDMP  was  developed  specifically  for  the  JRTC 
LTP,  it  contains  some  materials  tailored  explicitly  for  their  program.  It  is  also  rather 
heavily  light  Infantry  in  focus  and  examples,  although  the  materials  are  relevant  for 
mechanized  Infantry  and  other  branches.  The  program  provides  doctrinal  training  on 
staff  responsibilities  in  using  the  MDMP  model  and  additional  TTPs  are  provided  where 
appropriate  for  each  staff  position/section  to  include  the  same  thirteen  as  were  featured 
in  the  original  Battle  Staff  Training  Program  materials.  The  Brigade  Battle  Captain 
program,  based  on  JRTC-provided  information,  also  benefited  from  assistance  from  the 
Ranger  Training  Brigade,  the  U.  S.  Army  Engineer  School,  and  the  U.  S.  Army  Infantry 
School.  This  program  contains  TTPs  from  CALL  bulletins  and  newsletters,  and  from 
materials  in  use  at  the  Armor  and  Infantry  Advance  Courses  and  all  three  CTCs. 
Although  Brigade  Battle  Captain  was  designed  with  the  Infantry  in  mind,  the  materials 
are  equally  useful  for  Battle  Captains  at  battalion  level,  and  in  non-infantry  branches. 
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Purpose 


The  purpose  of  the  research  reported  here  was  to  perform  a  limited  assessment 
of  the  effectiveness  of  the  MDMP  and  Brigade  Battle  Captain  training  programs.  The 
overall  intent  was  to  ensure  that  the  programs  effectively  train  individuals  in  their  duties 
and  responsibilities.  Because  both  programs  include  materials  from  the  doctrinal 
manuals  as  well  as  TTPs  gathered  from  lessons  learned,  the  evaluation  examines  the 
programs  from  both  a  doctrinal  and  practical  standpoint.  This  is  especially  important  in 
the  case  of  the  Battle  Captain  program.  Frequently  when  there  is  no  formal  training 
available,  any  new  training  is  considered  better  than  no  training,  and  it  is  therefore 
deemed  effective.  This  research  considers  both  the  strengths  and  weaknesses  of  the 
two  courses.  This  assessment  should  provide  a  foundation  for  building  better  training  in 
the  future,  and  making  improvements  to  these  prototype  products. 

Method 


Subjects 

The  subjects  used  in  this  assessment  were  all  volunteers.  Their  ranks  ranged 
from  second  lieutenant  to  lieutenant  colonel  and  they  came  from  Active  Duty,  National 
Guard,  and  Army  Reserve  units.  Personnel  included  small  group  instructors  and 
students  from  the  Infantry  Officer  Advance  Course,  cadre  from  the  Ranger  Training 
Brigade,  the  36th  Engineer  Group,  and  the  75th  Ranger  Regiment.  Additional 
respondents  were  both  trainers  and  reserve  staff  personnel  from  the  48th  Infantry 
Brigade  (GA),  and  trainers  and  reserve  staff  from  the  38th  Infantry  Field  Training  Group 
(IN). 

Procedure 


The  original  design  of  this  evaluation  called  for  MDMP  knowledge  pre-  and  post¬ 
tests  to  measure  training  effectiveness,  followed  by  surveys  to  collect  participant 
opinions  on  the  value  of  the  courses.  However,  since  the  self-paced  training  took  an 
unpredictable  (individually  variable)  number  of  hours  to  complete,  the  unit  instructed  the 
officers  to  complete  the  training  “when  possible.”  Unfortunately,  several  of  the  test 
subjects  deployed  with  their  units  on  a  real  world  mission  prior  to  their  completion  of  the 
training.  The  pre-test/post-test  plan  was  therefore  abandoned  and  only  the  evaluation 
forms  were  used.  The  Course  Assessment  forms  are  shown  at  Appendix  A. 

For  both  evaluations,  the  procedures  were  the  same.  Officers  were  given  a  CD 
containing  the  program  and  instructed  to  complete  the  course  at  their  own  pace.  The 
researcher  kept  in  contact  with  the  participants  to  ensure  progress.  After  the  course 
had  been  completed,  the  test  subject  officers  filled  out  the  evaluation  forms  to  provide 
the  researcher  with  their  opinions  on  the  effectiveness  of  the  course  materials. 
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Results 


The  results  are  presented  separately  for  the  MDMP  course  and  the  Brigade 
Battle  Captain  course.  The  former  results  are  presented  first. 

MDMP 

Forty-seven  officers  rated  the  MDMP  course  using  the  Course  Assessment  form. 
Course  content  was  rated  using  a  five-point  scale  with  1  being  “very  effective”  and  5 
being  “very  ineffective”.  For  the  most  part,  the  course  content  was  rated  as  being 
effective.  There  were  no  negative  ratings  for  any  of  the  items.  Most  of  the  items  were 
rated  as  being  either  “very  effective”  (29%)  or  “generally  effective”  (53%).  Only  18%  of 
the  items  were  rated  as  being  only  “somewhat  effective”.  Table  1  provides  the  ratings 
for  the  specific  lessons  contained  within  the  MDMP  course. 


Table! 

MDMP  Course  Ratings  for  Individual  Lessons  (Percentages). 


Lesson  / 

Rating 

Very 

Effective 

Generally 

Effective 

Somewhat 

Effective 

Generally 

Ineffective 

Very 

Ineffective 

Receipt  of  Mission 

17 

75 

9 

0 

0 

Mission  Analysis 

6 

66 

28 

0 

0 

COA  Development 

13 

68 

19 

0 

0 

COA  Analysis 

17 

70 

13 

0 

0 

COA  Comparison 

38 

53 

9 

0 

0 

COA  Approval 

13 

79 

9 

0 

0 

Orders  Production 

38 

40 

21 

0 

0 

Note.  N  =  47. 


The  lesson  on  Course  of  Action  (COA)  Comparison  was  the  highest  rated  lesson 
with  38%  of  the  ratings  as  “very  effective”  and  53%  of  the  ratings  as  “generally 
effective.”  Most  of  the  comments  for  this  lesson  were  positive  (e.g.,  “good  techniques”). 
Mission  Analysis  was  the  lowest  rated  lesson  with  only  6%  of  the  ratings  as  “very 
effective”  and  28%  of  the  ratings  as  “somewhat  effective.”  The  negative  comments 
focused  on  the  lack  of  information  on  some  topics  and  too  much  information  on  other 
topics.  For  example,  several  officers  felt  that  there  was  too  little  emphasis  on 
Commander’s  Critical  Information  Requirements  ( CCIR )  and  the  development  of  the 
initial  Commander’s  Intent.  Risk  Assessment  and  Risk  Management  was  cited  as  a 
subject  that  was  covered  in  too  much  detail. 

Ratings  were  also  collected  for  the  content  areas  within  Mission  Analysis.  The 
survey  contained  more  detail  for  rating  this  lesson  because  mission  analysis  is  often  the 
area  of  the  MDMP  cited  as  weak  (e.g.,  NTC  Priority  Trends,  1996).  Within  Mission 
Analysis,  the  subject  rated  the  lowest  was  the  lesson  on  Identification  of  Specified  and 
Implied  Tasks  with  more  than  half  of  the  ratings  being  only  “somewhat  effective”. 

Typical  comments  were  that  the  subject  was  “too  basic.”  Risk  Assessment  was  ranked 
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highly  with  almost  half  of  the  ratings  being  “very  effective.”  This  rating  contrasts  with 
many  of  the  previous  comments  which  stated  that  there  was  too  much  emphasis  placed 
on  risk  assessment.  Although  over  half  the  ratings  were  “very  effective,”  23%  of  the 
ratings  were  “somewhat  effective."  It  may  have  been  that  the  individuals  rating  Risk 
Assessment  lower  felt  more  strongly  about  their  rating  than  those  rating  it  higher.  Table 
2  contains  the  rankings  for  all  of  the  sub-lessons  of  Mission  Analysis. 

Table  2. 

Mission  Analysis  Sub-lesson  Ratings  (Percentages). 


Sub-Lesson/ 

Rating 

Very 

Effective 

Generally 

Effective 

Somewhat 

Effective 

Generally 

Ineffective 

Very 

Ineffective 

Development  of  Mission 
Analysis  Products 

19 

43 

38 

0 

0 

Identification  of  Specified 
and  Implied  Tasks 

30 

17 

53 

0 

0 

Identification  of  Available 
Assets 

26 

23 

51 

0 

0 

Mission  Constraints  and 
Restrictions 

9 

60 

32 

0 

0 

Risk  Assessment  and 

Risk  Management 

47 

30 

23 

0 

0 

Mission  Analysis  Briefing 
Format 

17 

68 

15 

0 

0 

Commanders  Guidance 

17 

51 

32 

0 

0 

Second  Warning  Order 

26 

55 

19 

0 

0 

Practical  Exercises  for 
Mission  Analysis 

6 

57 

36 

0 

0 

Note.  N  =  47. 


Table  3  contains  the  ratings  of  the  TTPs  for  each  of  the  key  staff  members. 
Comments  tended  to  be  very  positive  such  as  “the  TTPs  were  the  best  part  of  the 
course.”  The  only  negative  comments  were  from  those  individuals  wanting  more  CTC 
rotation-based  TTPs,  i.e.,  more  information  that  would  help  them  during  their  CTC 
training  event. 

The  second  set  of  survey  questions  focused  on  how  well  the  officers  liked  the 
design  of  the  courseware  and  whether  they  would  use  it  for  training  other  staff 
personnel.  All  47  respondents  checked  that  the  courseware  was  “very  easy”  to  use. 
Forty-one  of  the  respondents  indicated  that  they  would  implement  the  courseware  into 
their  staff  training.  Of  the  six  saying  they  would  not  implement  the  courseware  into  their 
staff  training,  three  said  that  they  did  not  have  a  staff  and  one  stated  that  the  material 
was  too  focused  on  JRTC.  These  results  mirror  the  question  asking  the  respondents 
whether  the  courseware  was  relevant  to  their  unit.  Forty-one  responded  yes,  five 
responded  no  and  one  did  not  respond.  The  results  of  the  overall  training  ratings  are 
presented  in  Table  4. 
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Table  3. 

Ratings  for  TTPs  (Percentages). 


TTP/ 

Rating 

Very 

Effective 

Generally 

Effective 

Somewhat 

Effective 

Generally 

Ineffective 

Very 

Ineffective 

Overall  TTPs 

40 

57 

2 

0 

0 

Commander  TTPs 

51 

28 

6 

0 

0 

XO  TTPs 

53 

43 

4 

0 

0 

SI  TTPs 

36 

60 

4 

0 

0 

S2  TTPs 

51 

43 

6 

0 

0 

S3  TTPs 

26 

57 

17 

0 

0 

S4  TTPs 

47 

49 

4 

0 

0 

FSO  TTPs 

60 

34 

6 

0 

0 

Engineer  TTPs 

30 

66 

4 

0 

0 

Other  Staff  Members 

26 

70 

4 

0 

0 

Note.  N  =  47  except  for  commander,  N  =  40. 


Table  4. 

Ratings  of  Overall  Training  Received  on  MDMP  (Percentages). 


Statement 

Rating  (%) 

EXCEPTIONAL.  Training  is  above  expectations. 

0 

OUTSTANDING.  Most  required  information  can  be 
trained  with  very  few  modifications. 

87 

STANDARD.  Acceptable  training  capability. 

13 

MINIMAL  TRAINING  PROVIDED.  Major  modifications 
will  be  needed  to  provide  adequate  training. 

0 

POOR.  Cannot  train  because  of  poor  design  and 
content. 

0 

UNACCEPTABLE.  Provides  negative  training. 

0 

Note.  N  =  47. 


The  remaining  questions  elicited  opinions  on  the  best  and  worst  features  of  the 
training.  Some  respondents  focused  on  specific  features  of  the  courseware.  For 
example,  the  most  often  cited  positive  feature  was  the  user  friendliness  of  the  software. 
Negative  features  frequently  cited  were  the  length  of  time  to  exit  the  program  and  the 
lack  of  a  bookmark  feature  to  quickly  find  your  place  when  you  come  back.  Several 
respondents  requested  a  job  aid  feature  that  would  provide  blank  charts  to  fill  in  current 
data  while  the  user  is  in  the  program. 

Other  respondents  focused  on  course  content.  For  example,  positive  comments 
were  made  about  the  TTPs.  However,  more  detail  on  Mission  Analysis  (e.g.,  decisive 
point,  commanders’  intent,  key  terrain,  and  CCIR)  was  cited  as  content  that  should  be 
added  to  the  training  materials. 
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Sixty-one  officers  rated  the  Brigade  Battle  Captain  course  using  the  Course 
Assessment  Form  (see  Appendix  A).  Again,  the  course  content  was  rated  using  a  five- 
point  scale  with  1  being  “very  effective”  and  5  being  “very  ineffective.”  Like  the  MDMP 
course,  the  Brigade  Battle  Captain  course  content  was  primarily  rated  as  effective.  One 
sub  lesson  contained  within  the  Battle  Tracking  lesson  contained  two  negative  ratings 
(i.e.,  Battle  Rhythm,  “Generally  Ineffective”  3%).  Otherwise,  there  were  no  negative 
ratings  for  any  of  the  items.  Most  of  the  items  were  rated  as  being  either  “very  effective” 
(47%)  or  “generally  effective”  (44%).  Table  5  provides  the  ratings  for  the  specific 
lessons  contained  within  the  course. 


Table  5. 

Ratings  for  Brigade  Battle  Captain  Lessons  and  Sub-Lessons  (Percentages). 


38 

47 


Generally 

Effective 

75 

67 

52 

13 

26 

47 

41 

69 

49 

43 

75 


Somewhat 

Effective 

0 

13 

8 

10 

0 

0 

3 

7 

7 

15 

7 


Generally 

Ineffective 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


Very 

Ineffective 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


Rating  Very  Generally  Somewhat  Generally  Very 

Effective  Effective  Effective  Ineffective  Ineffect 

Overall  T TPs _ 25 _ 75 _ 0 _ 0 _ 0_ 

Job  Aids  (overall) _ 20 _ 67 _ 13 _ 0 _ 0_ 

Battle  Captain  Overview _ 39 _ 52 _ 8 _ 0 _ 0 

Defining  Battle  Captain _ 77 _ 13 _ 10 _ 0 _ 0_ 

Roles  &  Responsibilities _ 74 _ 26 _ 0 _ 0 _ 0 

Battle  Captain  Notebook _ 53 _ 47 _ 0 _ 0 _ 0 

Common  Problem  Areas _ 56 _ 41 _ 3 _ 0 _ 0_ 

Managing  Infomiation _ 24 _ 69 _ 7 _ 0 _ 0_ 

CCIR _ 44 _ 49 _ 7 _ 0 _ 0_ 

RFI _ 42 _ 43 _ 15 _ 0 _ 0_ 

Battle  Tracking  |  18  I  75  |  7  |  6  ^  0~ 


Battle  Rhythm _ 

Battle  Tracking 

Techniques _ 

Planner/Decision  Maker 


Role  in  Decision-Making 
Process 


Battle  Planning  Tools 
Issue  WARNO  &  FRAGO 


OPORD  Production 
TOC  Shift  OIC 


Shift  Change  Planning  & 
Briefing 


Briefing  Visitors 


TOC  Security _ _ _ _ 

Staff  Battle  Drills  38  1  42  |  20  1  0  1  0 

Note.  N  =  61.  RFI  =  Requests  for  Information;  WARNO  =  Warning  Order;  FRAGO  = 
Fragmentary  Order;  OPORD  =  Operations  Orders;  OIC  =  Office  in  Charge. 


The  highest  rated  sub  lessons  were  contained  within  the  Battle  Captain  Overview 
lesson.  Defining  Battle  Captain  was  rated  by  77%  of  the  respondents  as  “very  effective, 


while  74%  rated  Roles  and  Responsibilities  as  “very  effective.”  Comments  were 
consistent  with  the  ratings,  but  they  did  not  provide  additional  information  (e.g.,  “good 
lesson,”  “good  info”).  However,  for  the  major  lessons,  TOC  Shift  OIC  had  the  greatest 
percentage  of  respondents  (54%)  rating  it  as  “very  effective.” 

The  overall  lesson  receiving  the  fewest  “very  effective”  ratings  was  Battle 
Tracking.  It  received  only  18%.  Most  of  the  negative  comments  focused  on  the  need 
for  more  examples  and  more  TTPs.  Many  respondents  felt  that  this  important  section 
was  too  brief  and  too  generalized  in  content. 

Additional  questions  were  asked  about  the  general  effectiveness  of  the  course 
content  and  the  courseware  design.  Eighty-two  percent  of  the  respondents  indicated 
that  the  courseware  was  very  easy  to  use.  Ninety-five  percent  felt  that  the  course 
content  was  relevant  to  their  unit  and  that  they  would  incorporate  the  training  into  their 
unit  training.  Ninety-two  percent  indicated  that  the  design  of  the  courseware  enhanced 
learning  of  the  material.  Of  the  few  that  indicated  the  design  distracts  from  learning,  the 
most  common  complaint  was  that  the  scenes  on  the  slides  were  repetitive.  Some  also 
commented  that  the  graphics  were  slow  loading.  When  asked  about  the  overall  quality 
of  the  training,  69%  checked  that  the  training  was  “Outstanding.  Most  required 
information  can  be  trained  with  few  modifications.”  The  remaining  31%  checked 
“Standard.  Acceptable  training  capability.” 

The  remaining  three  questions  of  the  survey  asked  for  opinions  on  the  best 
features,  the  worst  features,  and  any  additional  training  content  which  should  be 
included  in  the  program.  Responses  to  the  question  on  what  they  felt  were  the  best 
features  were  consistent  with  the  lesson  ratings.  Most  respondents  indicated  that  the 
best  part  of  the  course  was  the  description  of  the  Battle  Captain  Roles  and 
Responsibilities.  The  question  pertaining  to  the  worst  features  elicited  two  main  areas, 
content  and  program  features.  For  content,  many  felt  that  there  was  too  much  time 
spent  on  Issue  Warning  Order  and  Operations  Orders  Production.  Most  felt  that  extra 
time  should  be  spent  on  Battle  Tracking  by  providing  examples  and  vignettes.  For 
program  features,  many  felt  that  the  program  loaded  too  slowly  and  that  there  should  be 
a  bookmark  feature  added. 

The  question  regarding  additional  features  needed  provided  a  variety  of 
responses.  The  most  frequently  mentioned  content  items  were  emergency  jump  TOC 
procedures,  key  staff  responsibilities,  vignettes  (historical),  and  samples  of  TOC 
configurations  and  TOC  SOPs.  The  program  features  most  requested  were  the 
bookmark  for  easy  access,  and  end  of  section  quizzes. 

Discussion 

Trained  and  efficient  battle  staffs  are  key  to  successful  mission  accomplishment 
in  combat  or  the  combat  training  provided  at  the  CTCs.  Many  apparent  training 
deficiencies  have  been  documented  from  lessons  learned  at  the  CTCs  and  described 
by  CALL.  As  noted,  many  of  the  shortcomings  experienced  by  battle  staffs  come  from 
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problems  in  execution  of  the  MDMP  and  with  battle  tracking.  The  MDMP  course, 
developed  to  reinforce  current  doctrine  as  well  as  to  provide  TTPs  from  lessons  learned, 
should  permit  staff  personnel  to  better  apply  the  MDMP  process.  Although  the  material 
was  available  already,  standardized  in  doctrinal  manuals,  staffs  were  unaware  of  how  to 
use  it.  This  course,  designed  to  be  self-paced,  should  help  staff  personnel  to  master 
the  process,  and  the  synchronization  required  of  a  staff.  For  the  Brigade  Battle  Captain, 
the  course  was  developed  to  provide  a  standardized  format  for  a  non-doctrinal  position. 
It  serves  that  purpose  and  provides  practical  TTPs,  tested  by  other  staffs,  to  assist  in 
the  process  of  training  the  new  Battle  Captain. 

Course  Evaluations 


Both  courses  were  well  received  by  the  soldiers  participating  in  the  evaluations. 
More  than  75%  rated  the  lessons  and  sub  lessons  as  being  “generally  effective”  or  “very 
effective.”  The  TTPs  included  were  seen  as  the  most  beneficial  aspects  of  the  courses. 
Because  there  is  no  formal  training  for  the  Battle  Captain,  another  highly  regarded  area 
of  training  was  the  Duties  and  Responsibilities  lesson  contained  in  the  Brigade  Battle 
Captain  Course. 

The  most  common  complaint  for  both  courses  was  the  lack  of  a  bookmark 
feature.  Many  respondents  felt  a  need  for  a  bookmark  feature  because  of  the  length  of 
the  courses.  The  feature  would  allow  students  to  return  to  the  place  in  the  course 
where  they  left  off  without  having  to  page  through  a  lot  of  previously  viewed  material. 
Another  feature  that  many  of  the  respondents  wanted  to  see  added  was  an  interactive 
practical  exercise.  They  felt  that  an  exercise  using  historical  data  would  allow  the 
student  to  practice  the  material  learned  in  the  programs. 

Some  of  the  other  areas  that  were  requested  had  to  do  with  how  the  various  staff 
members  interact.  Because  different  personnel  reviewed  the  MDMP  and  the  Brigade 
Battle  Captain  courses,  the  test  subjects  were  unaware  that  other  courses  with  other 
TTPs  might  contain  the  information  they  wanted.  For  example,  there  were  a  few 
respondents  who  indicated  a  need  for  greater  definition  of  other  battle  staff  member 
duties  in  their  review  of  the  Brigade  Battle  Captain  course.  However,  more  than  a 
general  discussion  is  outside  the  scope  of  the  course  which  was  intended  to  provide 
only  a  general  overview.  Personnel  desiring  more  information  would  be  referred  to  the 
appropriate  field  manuals,  to  the  MDMP  Course,  or  to  the  Battle  Staff  Training  System 
courses  for  more  detailed  information. 


Conclusion 

The  MDMP  and  the  Brigade  Battle  Captain  computer  based  instruction  courses 
were  well  received  by  the  evaluation  respondents.  In  fact,  ARI  had  received  requests 
for  the  material  even  prior  to  the  external  evaluation.  This  was  no  doubt  in  large  part 
due  to  the  fact  that  there  has  been  a  lack  of  available  training  material  for  these  two 
subject  areas.  The  design  of  future  courses  would  be  enhanced  by  a  means  for 
returning  to  a  lesson  if  the  student  exits  the  training  before  completion.  In  addition, 
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training  would  be  enhanced  with  interactive  practical  exercises  using  historical  data 
from  CTCs  and  real  world  examples.  In  sum,  however,  and  based  on  the  replies  of  the 
participants,  these  prototype  courses  are  a  good  first  step  in  training  both  the 
procedures  and  steps  of  the  military  decision-making  process,  and  the  duties  and 
responsibilities  of  the  Battle  Captain. 
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APPENDIX  A 


MDMP  and  Battle  Captain  Course  Evaluation  Forms 


A-1 


Military  Decision  Making  Course  Assessment 


Duty  Position 


Time  in  current  duty  position 


Unit  Type_ 


1 .  Use  the  following  scale  to  rate  how  effectively  you  think  the  courseware  trains  the  following 
tasks. 

1  =  Very  effective  2  =  Generally  effective  3  =  Somewhat  effective 
4  =  Generally  ineffective  5  =  Very  ineffective  N/A  =  Did  not  train 


Rating 


Comment 


Tactics  Techniques  and  Procedures  (TTP) 
(overall)  _ 


Commander  TTPs 


XOTTPs 


SI  TTPs 


S2  TTPs 


S3  TTPs 


S4  TTPs 


FSO  TTPs 


Engineer  TTPs 


Other  staff  Members 


Receipt  of  Mission  (overall  lesson) 


Mission  Analysis  (overall  lesson) 


Development  of  mission  analysis  products 
(e.g.,  R&  S  plan,  Enemy  Situation 
Templates) _ 


Identification  of  specified  &  implied  tasks 


Identification  of  available  assets 


Identification  of  mission  constraints  & 
restrictions  _ 


Risk  assessment  &  risk  management 


Mission  Analysis  briefing  format 


Commander’s  guidance 


Second  warning  order 


Practical  exercises  for  Mission  Analysis 


COA  Development  (overall  lesson) 


COA  Analysis  (overall  lesson) 


COA  Comparison  (overall  lesson) 


COA  Approval  (overall  lesson) 


Orders  Production  (overall  lesson) 


2.  How  easy  is  the  courseware  to  use? 


_ very  easy 

_ too  difficult 

3.  Will  you  implement  the  courseware  into  your  staff  training? 
_ yes 

if  so,  who  will  use  it? _ 

where  would  it  be  used  in  your  unit? 


_ no 

4.  Mark  the  sentence  that  best  describes  the  training  you  received  on  MDMP. 

_ Exceptional.  Training  is  above  expectations. 

_ Outstanding.  Most  required  information  can  be  trained  with  very  few  modifications. 

_ Standard.  Acceptable  training  capability. 

_ Minimal  training  provided.  Major  modifications  will  be  needed  to  provide  adequate 

training. 

_ Poor.  Cannot  train  because  of  poor  design  and  content. 

_ Unacceptable.  Provides  negative  training. 

5.  Were  the  examples  and  TTP  used  relevant  to  your  unit? 

_ yes 

_ no,  what  is  needed? _ 


6.  Quality  of  the  courseware.  Please  check  the  appropriate  box. 


Media 

Enhance 

s 

Distracts 

Video  clips 

Slides 

Graphics 

Audio 

Button  layout 

7.  What  are  the  features  of  the  computer  that  you  used  to  run  the  MDMP  software?  (e.g.,  486 
66MHz  with  16MB  RAM  and  4X  CD  ROM  drive). 


A-3 


8.  What  were  the  best  features  of  the  MDMP  training? 


9.  What  were  the  worst  features  of  the  MDMP  training? 


10.  What  would  you  like  added  to  the  MDMP  training? 


A-4 


Brigade  Battle  Captain  Course  Assessment 


Duty  Position 


Time  in  current  duty  position 


Unit  Type . 


1 .  Use  the  following  scale  to  rate  how  effectively  you  think  the  courseware  trains  the  following 
Battle  Captain  duties  and  responsibilities. 

1  =  Very  effective  2  =  Generally  effective  3  =  Somewhat  effective 
4  =  Generally  ineffective  5  =  Very  ineffective  N/A  =  Did  not  train 


Rating 


Comment 


Tactics  Techniques  and  Procedures  (TTPs) 
(overall) _ 


Job  Aids  (overall) 


Battle  Captain  overview  (overall  lesson) 


Defining  the  Battle  Captain 


Roles  and  Responsibilities 


Battle  Captain  Notebook 


Common  Problem  areas 


Managing  Information  (overall  lesson) 


Commanders  Critical  Information 
Requirements  _ 


Requests  for  Information 


Battle  Tracking  (overall  lesson) 


Battle  Rhythm 


Battle  Tracking  Techniques 


Planner/Decision-maker  (overall  lesson) 


Role  in  Decision-Making  Process 


Battle  Planning  Tools 


Issue  WARNO  &  FRAGO 


OPORD  Production 


TOC  Shift  OIC  (overall  lesson) 


Shift  Change  Planning  &  Briefing 


Briefing  Visitors 


TOC  Security 


Staff  Battle  Drills 


2.  How  easy  is  the  courseware  to  use? 

_ very  easy 

_ too  difficult 

3.  Will  you  implement  the  courseware  into  training  your  Battle  Captain  and  other  staff  members? 

_ yes 

if  so,  who  will  use  it? _ 

where  would  it  be  used  in  your  unit? 


_ no 

4.  Mark  the  sentence  that  best  describes  the  quality  of  training  contained  in  the  Battle  Captain 
Course. 

_ Exceptional.  Training  is  above  expectations. 

_ Outstanding.  Most  required  information  can  be  trained  with  very  few  modifications. 

_ Standard.  Acceptable  training  capability. 

_ Minimal  training  provided.  Major  modifications  will  be  needed  to  provide  adequate 

training. 

_ Poor.  Cannot  train  because  of  poor  design  and  content. 

_ Unacceptable.  Provides  negative  training. 

5.  Were  the  examples  and  TTP  used  relevant  to  your  unit? 

_ yes 

_ _  no,  what  is  needed? _ _ 


6.  Quality  of  the  courseware.  Please  check  the  appropriate  box. 


Media 

Enhance 

s 

Distracts 

Slides 

Graphics 

Audio 

Button  layout 

8.  What  are  the  features  of  the  computer  that  you  used  to  run  the  software?  (e.g.,  486  66MHz 
with  16MB  RAM  and  4X  CD  ROM  drive). 


A-6 


8.  What  were  the  best  features  of  the  Battle  Captain  training? 


9.  What  were  the  worst  features  of  the  Battle  Captain  training? 


10.  What  would  you  like  added  to  the  Battle  Captain  training? 


A-7 


